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IV 


Foreword 


The  Small  Business  Innovation  Research  (SBIR)  project,  “Training  Adaptability  in 
Digital  Skills,  Phase  II,”  was  conducted  under  contract  number  DASW01-01-C-005  by  Aptima, 
Inc.  and  the  Group  for  Organizational  Effectiveness,  Inc.  (gOE)  with  mentorship  by  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  Advanced  Training 
Methods  Research  Unit.  The  goal  of  the  project  was  to  develop  a  marketable  training  program 
that  would  increase  training  transfer  from  one  technology  system  or  version  to  another. 

ARI  has  a  research  program  focusing  on  how  to  best  train  the  digital  skills  necessary  for 
increased  battlefield  digitization.  The  concept  is  to  design  training  that  will  help  personnel 
benefit  from  the  full  potential  of  improved  technology.  One  way  to  accomplish  that  objective  is 
to  train  adaptability  as  a  key  aspect  of  training  for  successful  battlefield  digitization. 

The  purpose  of  the  Aptima  and  gOE  two-phase  SBIR  effort  was  to  increase  the  degree  to 
which  soldiers  are  able  to  apply  classroom/computer-mediated  training  on-the-job  and  to 
capitalize  on  their  present  digital  skills  in  the  acquisition  of  newer  and  changing  digital  skills.  In 
combination  with  other  research  being  done  at  ARI,  this  work  will  provide  part  of  a  foundation 
for  training  adaptability  in  digital  skills.  This  report  is  addressed  primarily  to  training  managers, 
developers,  and  system  designers. 


Acting  Technical  Director 
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TRAINING  ADAPTABILITY  IN  DIGITAL  SKILLS:  THE  LEARNING  SKILLS  BRIDGE 

(LSB)  LEARNING  ACCELERATOR 


Executive  Summary 


Research  Requirement: 

Given  the  rapidity  by  which  warfighting  digital  technology  is  being  updated  and 
replaced,  successful  warfighters  will  be  those  who  possess  a  solid  foundation  for  adaptive  digital 
skills  that  goes  beyond  abilities  to  utilize  specific  tools  and  applications  and  includes  deep 
knowledge  of  digital  concepts  that  are  relevant  in  every  context  of  use.  The  requirement  for  this 
Small  Business  Innovation  Research  (SBIR),  Phase  II  project  was  to  pick  an  exemplar 
technology  system  and  show  how  a  training  product  could  increase  the  degree  to  which  soldiers 
are  able  to  apply  classroom/computer-mediated  training  on-the-job  and  to  make  use  of  their 
current  digital  skills  in  the  acquisition  of  new  and  changing  digital  skills. 


Procedure: 

Aptima,  Inc.  and  the  Group  for  Organizational  Effectiveness,  Inc.  capitalized  on  the  basic 
theories  of  learning  transfer  to  create  the  Learning  Skills  Bridge  (LSB),  a  web-based  two-module 
training  package  designed  to  increase  the  soldier’s  digital  skill  adaptability.  The  first  module 
(basic  computer  knowledge),  was  created  by  working  with  computer-technology  and. 
leaming/training  subject  matter  experts  (SMEs)  to  identify  the  basic  computer  knowledge  that  is 
essential  to  ensure  learning  transfer  to  novel  or  changing  technologies.  Through  an  iterative 
process  of  content  development  and  revision,  the  researchers  were  able  to  identify  specific 
questions  and  informational  material  that  would  facilitate  the  learning  of  computer  knowledge 
essential  for  such  transfer.  The  second  module  of  the  LSB  training  program  (known  as  the 
“bridge”  component)  was  developed  through  interviews  and  discussions  with  operational  SMEs. 
With  this  information,  content  was  produced  that  encourages  learning  transfer  of  the  basic 
computer  knowledge  to  the  computer  system  of  interest,  the  Advanced  Field  Artillery  Tactical 
Data  System  (AFATDS),  by  illustrating  the  specific  relationship  between  the  basic  computer 
knowledge  and  the  target  technology  (i.e.,  AFATDS).  The  LSB  was  pilot  tested,  revised,  and 
eventually  used  to  collect  performance  data  for  soldiers  participating  in  AFATDS  training  at  Fort 
Sill,  OK.  In  this  final  data  collection,  participants  were  pre-tested  on  their  knowledge  of 
AFATDS  network  topics  and  AFATDS  visualization  and  mapping  topics.  Next  they  were  given 
the  LSB  training  package  and  told  to  go  through  both  the  Networks  section  and  the  Visualization 
and  Mapping  section.  Then  they  were  given  a  post-test  to  determine  whether  the  training  on  the 
basic  computer  knowledge  resulted  in  learning  transfer  to  the  target  system.  Sample  results  from 
the  Fort  Sill  data  collection  are  briefly  reviewed  below. 


Findings: 

The  findings  from  this  data  collection  effort  suggest  that  training  designed  to  increase 
basic  computer  knowledge  resulted  in  learning  transfer  to  the  AFATDS,  the  target  computer 
system.  Specifically,  the  LSB  training  eliminated  any  group  differences  based  on  prior  computer 
experience,  and  thus  compensated  for  deficient  computer  experience.  It  was  also  found  that 
participants  scored  significantly  higher  on  the  measures  of  AFATDS  networks  and  AFATDS 
visualization  and  Mapping  after  LSB  training  than  they  did  before  the  training.  Further,  88.8% 
of  participants  reported  that  the  LSB  training  improved  their  understanding  of  AFATDS.  These 
results  may  support  the  notion  that  generalizable,  transferable  digital  skills  taught  in  the  context 
of  device-  and  job-specific  goals  (such  as  the  LSB  training  program)  has  promise  in  increasing 
adaptability  in  the  use  of  those  digital  skills.  It  must  be  noted  that  this  was  an  initial  study 
incorporating  a  quasi-experimental  design.  As  such,  further  studies  incorporating  a  control 
group  are  needed  before  more  firm  conclusions  can  be  drawn. 


Utilization  of  Findings: 

The  current  version  of  the  LSB  is  ready  for  use  as  a  learning  accelerator  for  AFATDS 
training.  In  the  current  version  of  LSB,  trainees  may  complete  sections  in  an  order  and  timing  of 
their  choice.  As  such,  training  times  may  vary.  None  the  less,  allowing  trainees  to  complete 
relevant  sections  of  the  LSB  training  before  AFATDS  training  may  assist  trainees  to  complete 
the  AFTDS  training  in  less  time  while  learning  and  retaining  more  of  the  content.  AFATDS 
trainers  may  spend  less  time  covering  base  computer  knowledge  and  low-level  AFATDS 
knowledge,  so  they  can  focus  more  on  advanced  AFATDS  knowledge  and  skills.  More 
important  than  the  AFATDS  application,  however,  the  LSB  training  program  was  developed 
such  that  it  can  be  easily  re-configured  to  be  appropriate  for  any  target  system  that  requires  basic 
computer  skills.  For  example,  all  branches  of  the  armed  forces  are  experiencing  the  same 
technology  challenges,  and  the  police  and  the  FBI  are  both  experiencing  an  incredible  increase  in 
technology  use.  With  relatively  little  development,  the  LSB  can  be  revised  to  be  used  for  any  of 
these.  With  more  resources,  the  LSB  can  be  developed  to  handle  other  basic  knowledge  -  such 
as  basic  reading  skills  -  that  are  needed  for  other  target  platforms. 
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Training  in  Adaptive  Digital  Skills  Is  Critical 

This  two-phase  Small  Business  Innovative  Research  (SBIR)  project  addressed  a  critical 
Army  need:  training  soldiers  to  adapt  digital  skills  to  rapidly  evolving  technology.  It  is  not 
sufficient  for  soldiers  simply  to  learn  how  to  operate  today’s  digital  equipment.  Soldiers  must  be 
able  to  quickly  transfer  digital  knowledge  and  skills  to  set-up,  use,  and  maintain  new  software 
and  new  equipment. 

The  concept  of  Network-Centric  Warfare  is  rapidly  becoming  a  reality  for  today’s 
modem  military.  While  there  are  many  variations  in  definition,  some  common  attributes  of  this 
concept  include  the  following:  multiple  information  inputs,  a  collaborative  approach  at  multiple 
operational  levels  for  information  processing  and  decision  making,  choice  of  multiple  weapons 
platforms/resources,  and  a  larger  amount  of  real-time  self-sequencing  to  accomplish  tasks  at  a 
greater  speed  (Siebold  &  Benton,  2003).  Many  argue  that,  with  this  changing  nature  of  warfare, 
information  dominance  is  the  key  by  which  we  will  maintain  our  military  advantage  (Siebold  & 
Benton,  2003).  As  such,  military  technology  has  been  revolutionized  in  an  attempt  to  facilitate 
the  best  use  of  information.  In  practically  every  operational  context — from  simulation-based 
training,  to  digital  collaborative  planning,  to  real-time  video  conferencing  on  the  battlefield— the 
tools  used  by  the  modem  “digital  warrior”  are  networked  and  software  based.  Digital  equipment 
and  communication  evolve  rapidly.  Shortened  supplier  design/production  cycles  and  uniquely 
fast  improvements  in  software  and  computer  components  mean  that  new  equipment  and 
techniques  are  fielded  with  unprecedented  speed.  The  stability  of  the  military  environment,  and 
the  concomitant  predictability  of  tasks  and  equipment,  has  become  a  thing  of  the  past.  This 
means  that  the  “half-life”  of  equipment  training  is  short,  so  that  a  soldier  may  face  the  need  to 
master  new  equipment  in  the  field.  In  addition,  Army  personnel  may  have  to  work  under  an 
intense  optempo,  under  which  learning  conditions  are  not  optimal.  Thus,  for  Network-Centric 
Warfare  to  be  a  success,  adaptability  in  learning  to  use  equipment  requiring  digital  skills  under 
difficult  conditions  is  essential. 

In  such  digital  environments,  the  successful  warfighters  are  those  who  possess  a  solid 
foundation  of  adaptive  digital-literacy  that  goes  beyond  abilities  to  utilize  specific  tools  and 
applications,  and  includes  deep  knowledge  of  digital  concepts  that  are  relevant  in  every  context 
of  use.  Whether  faced  with  new  tools,  incremental  improvements  of  familiar  tools,  or  the  same 
tools  in  new  contexts  (e.g.,  from  a  CONUS  simulation  setting,  to  a  theater  of  war,  or  drug  ops 
application),  the  warrior  will  be  expected  to  apply  what  he  or  she  already  knows  to  quickly  come 
up  to  speed  on  new  tools,  often  with  “just-in-time”  rapidity.  Ensuring  this  level  of  knowledge 
transfer  will  require  new  approaches  to  training,  focused  on  providing  a  deep  foundation  of 
digital  skills  and  knowledge,  and  a  set  of  “tools”  and  strategies  that  the  warfighters  can  use  to 
apply  what  they  already  know  to  actively  engage  and  learn  new  tools.  In  fact,  one  of  the  major 
drivers  for  the  Objective  Force,  Task  Force  TLS  (“Training,”  “Leadership,”  and  “Soldier”), 
recognizes  that  the  role  of  the  soldier  will  change  dramatically  because  of  changes  in  technology, 
and  that  this  will  result  in  the  need  for  an  increase  in  adaptive  skills. 

The  U.S.  Army  Research  Institute  (ARI)  has  a  research  program  focusing  on  how  to  best 
train  the  digital  skills  necessary  for  battlefield  digitization.  The  concept  is  to  design  training  that 


1 


will  help  personnel  benefit  from  the  lull  potential  of  improved  technology.  One  way  to 
accomplish  that  objective  is  to  train  adaptability  as  a  key  aspect  of  training  for  successful 
battlefield  digitization.  The  framework  ARI  chose  to  organize  their  plan  of  attacking  this 
problem  consists  of  five  topics  (see  Figure  1),  each  emphasizing  a  different  area  in  need  of 
research  (Moses  2001). 

As  highlighted  in  Figure  1,  ARI  believes  that  training  adaptability  is  a  key  aspect  of 
training  for  successful  battlefield  digitization.  Moses  (2001)  suggests  that  “Soldiers  will  have  to 
go  beyond  minimal  operation  of  digital  workstations  to  be  effective”  (p.  17)  and  that  training 
must  shift  its  perspective  from  training  in  “what  to  think”  to  training  in  “how  to  think.”  A  key 
goal  identified  in  Moses  (2001)  for  digital  skills  training  is  to  “develop  training  methods  and 
strategies  that  promote  adaptive/flexible  and  innovative  behavior"’  (p.  18).  Schaab  and  Moses 
(2001)  address  ways  to  assist  soldiers  in  applying  digital  skills  in  a  wide  variety  of  military 
settings  by  having  trainers  “present  instruction  in  the  context  of  realistic  situations”  (p.  6)  and 
“integrate  digital  systems  into  everyday  work  routines  whenever  possible”  (p.  9).  The  purpose  of 
the  Aptima/gOE  two  phase  effort  was  to  increase  the  degree  to  which  soldiers  are  able  to  1) 
apply  classroom/computer-mediated  training  on-the-job  and  2)  capitalize  on  their  present  digital 
skills  in  the  acquisition  of  newer  and  changing  digital  skills. 


Determine  Digital  Task  Training  Requirements 

How  should  training  requirements  analyses  change  for 
digital  tasks  and  for  hybrid  digital-conventional  environments 
to  determine  the  training  needed  for  producing  high 
performance  soldiers? 


Training  Delivery  on  Demand 

What  training  should  be  done  now,  when 
and  where  using  soldier-centered 
cfistributed/digital  environments  JPSJ 


Assess  Skill  Levels  of 
Digital  Soldiers 

What  measurement/assessment 
and  feedback  tools  are  needed  to 
ensure  that  required  skill  levels 
are  attained  and  retained? 


Train  Adaptability 

What  training  methods 
work  best  to  prepare 
soldiers  for  frequent 
software/equipment 
changes  combined  with 
ad-hoc  and  varied 
missions  (including  combat 
and  special  assignments)? 


Prepare  Digitally-Linked  Teams 

What  methods  work  best  to  prepare  leaders  and  to 
train  widely  dispersed  soldiers  linked  as  teams  and 
units  through  digital  networks? 


Figure  1.  Five  Training  Challenges  for  Digitization  (Moses,  2001)  [emphasis  added] 
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The  Aptima/gOE  Solution 

In  Phase  I  of  this  project,  Aptima,  Inc.  and  the  Group  for  Organizational  Effectiveness 
(gOE)  laid  the  groundwork  for  an  innovative  digital-skills  training  package  designed  to  increase 
the  adaptability  of  digital  skills.  In  Phase  II  the  Aptima/gOE  team  developed  the  Learning  Skills 
Bridge  (LSB)  “learning  accelerator”  training  program.  The  LSB  is  a  web-based  training  package 
that  uses  asynchronous  computer-based  instruction  and  sound  pedagogical  theory  to  train  basic 
computer  knowledge  and  its  application  to  job-specific  requirements.  Generalizable,  transferable 
digital  skills  are  not  taught  in  a  vacuum  but  instead  are  learned  in  the  context  of  device-  and  job- 
specific  goals.  This  approach  was  developed  based  on  the  literature  on  learning  and  transfer  of 
training.  Specifically,  there  exist  two  major  leaming/training  theories  in  relation  to  skill  transfer 
to  the  job  environment.  The  first  theory,  identical  elements  theory,  is  adopted  from  classical 
learning  theory  and  states  that  transfer  will  occur  as  long  as  there  are  the  same  elements  in  the 
training  and  application  environments.  The  second  theory,  principle  theory,  states  that  training 
should  focus  on  the  global  principles  necessary  to  learn  a  task  so  that  the  learner  may  apply  them 
to  a  novel  situation  (Goldstein  &  Ford,  2002).  While  training  designers  typically  adopt  one  of 
these  two  learning  theories  in  which  to  base  their  training  programs,  the  LSB  training  program 
capitalizes  on  the  strength  of  both  theories. 

The  LSB  training  package  has  two  “modules”  -  one  a  general  computer  module,  the 
other  a  device  specific  “bridge”  module.  Through  the  general  computer  training  we  adhere  to 
“principle  theoiy”  -  the  instruction  and  assessment  are  designed  to  provide  a  comprehensive 
global  understanding  of  important  computer/technological  issues.  The  use  of  the  bridge  module 
allows  us  to  incorporate  the  “identical  elements  theory.”  The  content  of  the  bridge  portion  is 
designed  to  be  specific  to  the  technological/applications  elements  that  the  trainee  will  be 
expected  to  use  on  the  job.  Consequently,  the  LSB  is  designed  to  both  increase  the  effectiveness 
of  fixture  training  and  increase  the  degree  to  which  trained  skills  are  applied  on  the  job.  The 
general  device  knowledge  training  will  ensure  true  adaptability  when  going  from  one  computer 
system  to  another,  while  the  device-specific  training  will  allow  the  trainees  to  take  full  advantage 
of  the  general  training  in  a  specific  situation. 

The  LSB  provides  today’s  AFATDS  operator  with  a  deep  foundation  of  computer 
knowledge,  relates  this  foundational  knowledge  to  his  target  computer  system,  and  permits  the 
delivery  of  training  content  directly  to  operational  settings  using  modem  distance-learning 
methods.  This  training  program  is  also  a  commercial  “learning  accelerator”  that  can  be  adapted 
to  prepare  other  users  of  different  target  systems.  The  LSB  training  provides  a  foundation  of 
general  digital-tool  concepts  and  strategies  for  mapping  these  concepts  to  new  applications.  This 
training  provides  a  deeper  base  of  knowledge  and  cognitive  learning  strategies  to  insure  rapid 
transfer.  The  remainder  of  this  report  describes  how  the  LSB  tool  was  developed  and  tested. 

Developing  the  LSB  Tool 
Backend 

The  major  requirement  of  the  Learning  Skills  Bridge  (LSB)  program  that  guided 
development  was  to  present  information  in  a  non-linear  fashion  so  that  the  user’s  movement 
through  the  program  is  determined  by  his/her  interests  and  expertise  instead  of  a  predetermined 
path.  To  allow  for  this,  we  created  a  software  product  made  up  of  many  components:  a  backend 
data  server,  a  web  server  for  serving  dynamically  created  web  pages,  dynamic  script  execution 
for  generating  web  pages,  and  the  student  side  user  interface.  The  software  is  organized  into  four 
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areas:  data,  student,  administration,  and  content  authoring  as  shown  in  Figure  2.  Each  of  these 
areas  is  explained  in  more  detail  below. 


STUDENTS 

assa 


Figure  2.  LSB  System  Architecture 


Data.  The  “data”  portion  of  the  program  provides  the  basic  connectivity  to  the  content 
and  student  data  that  resides  in  the  database.  An  Application  Programming  Interface  (API)  was 
used  to  develop  the  PHP  Hypertext  Preprocessor  (PHP;  http://www.php.net)  scripting  language 
that  makes  communicating  with  MySQL  (http://www.mysql.com)  based  databases  very  easy. 
MySQL  based  databases  are  relational  database  systems  that  store  data  in  the  form  of  related 
tables.  This  type  of  relational  database  system  was  needed  to  ensure  the  flexibility  needed  for 
the  LSB.  The  database  schema  was  designed  using  upfront  requirements  analysis  and  then  fine- 
tuned  via  an  iterative  process  during  development. 

Student.  The  “student”  portion  of  the  program  is  a  large  set  of  dynamically  generated 
web  pages  that  are  controlled  by  the  program.  These  are  the  pages  with  which  the  student 
interacts.  The  responses  of  the  student  govern  the  generation  of  the  next  page. 

Administration.  The  “administration”  portion  of  the  program  presents  the  local 
administrator  with  options  for  adding  students  and  generating  reports.  The  administrator  also 
adds  program  specific  “bridge”  content,  controls  the  number  of  questions  a  student  must  answer 
for  each  concept,  as  well  as  which  concepts,  topics,  and  categories  to  include  in  the  customized 
training  program. 

Authoring.  The  “content  authoring”  portion  of  the  program,  one  part  of  the  “global 
administration”  level,  allows  for  the  addition  of  content,  both  instructional  and  questions. 

Software  Used 

The  Aptima  software  developers  designed  the  LSB  system  to  use  a  Structured  Query 
Language  (SQL)  database  as  the  backend  storage  system.  For  its  simplicity  and  speed,  the 
MySQL  Open  Source  product  is  used.  The  user-side  interface  is  created  using  the  web  scripting 
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language  PHP  for  the  majority  of  the  work  and  some  Javascript  programs  are  mixed  in  for 
various  visual/programmatic  effects. 

The  software  development  team  used  standard  development  workstations  running  Linux 
and  Windows  2000/XP  supplied  with  the  appropriate  software.  Each  developer  created  a 
miniature  version  of  the  actual  website  for  testing  purposes.  Each  workstation  ran  with  the  same 
versions  of  the  servers  as  the  actual  target  server. 

The  source  control  product  (CVS;  http://www.cvs.org)  was  used  to  store  and  manage  the 
projects  source  code.  The  use  of  this  tool  creates  an  environment  more  conducive  to  team 
development.  During  the  development  process  Aptima  tracked  the  software  issues  using  email 
and  a  simple  database.  Product  reviews  were  held  regularly  during  the  phase  leading  up  to 
deployment  with  the  team  reporting  issues  and  verifying  fixes. 

The  software  is  deployed  on  a  server  running  Linux,  Apache,  and  MySQL. 

Content 

In  developing  the  LSB  tool,  our  team  defined  a  set  of  content  areas.  The  information  to 
be  covered  in  the  tool  included  a  variety  of  high-level  computer  skill  categories  (e.g.,  computer 
networks)  with  lower-level  topics  (e.g.,  network  configuration)  and,  at  the  lowest  level,  concepts 
(e.g.,  bus  configuration).  After  initially  identifying  categories  and  their  topics  and  concepts 
considered  essential  for  basic  computer  understanding,  computer  experts’  opinions  on  these 
topics  were  gathered. 

The  basic  computer  skills  categories  realized  after  this  survey  are  as  follows: 

•  Computer  Networks 

•  Electronic  Communications 

•  Components 

•  Operating  Systems 

•  Visualization/Mapping 

•  Application  Software 

•  User  Interface 

Each  of  these  content  areas  was  then  populated  along  a  generic  structure  as  follows.  For 
each  category  there  was  an  overall  introduction.  Then,  for  each  topic  within  that  category,  there 
was  a  more  focused  introduction  for  that  specific  topic.  Finally,  for  each  concept  within  each 
topic  there  were  instructions  and  corresponding  questions.  The  categories,  topics  and  concepts 
were  initially  ordered  according  to  priorities  identified  by  the  computer  experts.  This  suggested 
order  can  be  easily  altered  as  needed. 

The  category,  “Visualization/Mapping,”  may  not  be  as  easily  perceived  as  part  of  the 
traditional  “computer  skills”  domain  as  the  other  categories  contained  in  the  program.  The 
rationale  for  the  inclusion  of  this  content  area,  however,  was  that  one  of  the  most  unique  and 
important  strengths  of  computer  technology  is  the  display  of  electronic  maps  and  graphics. 
Thus,  while  “mapping  and  visualization”  could  be  seen  as  two  of  a  myriad  of  possible  computer 
applications,  at  the  same  time  they  leverage  computer  technology  so  effectively  that  they  can 
almost  be  considered  an  extension  of  the  user  interface.  In  particular,  military  applications  use 
maps  and  visualization  so  frequently  that  we  considered  these  to  be  part  of  the  “digital  skills” 
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domain.  For  example,  to  “zoom”  in  or  out  on  a  map  is  a  critical  ability  for  many  digital 
applications. 

Introductions  and  Instructions 

Once  the  categories  were  selected,  the  team  authored  the  introductory  and  instructional 
materials  for  each  topic  through  an  iterative  authoring  and  reviewing  process.  A  lead  author  was 
assigned  for  each  category.  This  individual  used  online  and  hard  copy  resources  to  write  an 
initial  draft  of  the  introductory  and  instructional  materials  for  the  category,  specifically  creating 
content  for  each  of  the  topics  and  concepts.  During  this  process  of  research  and  authoring,  the 
author  would  often  identify  gaps  in  the  current  list  of  topics  and  concepts  for  a  category.  The 
author  would  create  additional  topics  or  concepts  to  fill  these  gaps  as  the  situation  required. 

Additionally,  die  author  would  segment  the  content  into  basic  and  advanced  concepts. 
Originally  it  was  intended  that  every  concept  would  have  both  basic  and  advanced  material; 
however,  during  the  authoring  process  it  became  clear  that  some  concepts  were  inherently  basic, 
while  others  were  inherendy  advanced,  and  a  smaller  subset  could  be  broken  into  both  basic  and 
advanced.  As  such,  the  team  decided  to  alter  the  software  requirements  to  allow  concepts  to 
have  only  basic,  only  advanced,  or  both  basic  and  advanced  material.  The  distinction  between 
basic  and  advanced  content  was  relatively  transparent  to  the  user;  however,  the  sections  were 
organized  to  encourage  the  user  to  work  through  the  basic  material  first  before  reaching  more 
advanced  concepts.  Once  a  complete  draft  of  a  category  was  complete,  the  draft  was  passed 
through  a  two-tier  review  process.  In  the  first  tier  of  the  review  process,  the  content  was 
reviewed  for  its  technical  accuracy.  In  the  second  tier  of  the  review  process,  the  content  was 
reviewed  for  clarity  and  structure,  with  an  eye  towards  making  die  content  clear  for  individuals 
who  are  likely  to  have  minimal  experience  with  the  topic.  The  second  tier  reviewer  also  focused 
on  providing  definitions  to  the  reader  for  potentially  unfamiliar  terms.  These  definitions  were 
created  from  combining  sources  from  the  web;  http://whatis.techtarget.com/. 
http://www.webopedia.com/.  and  The  American  Heritage  College  Dictionary:  Third  Edition.  The 
definitions  were  created  to  apply  across  the  computer  skills  categories.  Upon  completion  of  the 
second  tier  of  review,  the  content  was  converted  into  an  HTML  format  and  loaded  into  the 
database  to  be  made  available  through  the  tool. 

Throughout  the  authoring  and  review  process,  the  team  worked  to  maintain  consistency 
in  the  content,  providing  training  that  was  structured  in  a  logical  manner  and  supplemented  with 
information  that  would  make  it  easy  to  understand  for  novice  users.  Whenever  possible,  the 
authors  and  reviewers  identified  topics  that  could  be  accompanied  by  pictures  and  diagrams  to 
facilitate  a  student’s  understanding  of  the  concept.  The  author  would  create  a  diagram  that 
would  then  be  passed  on  to  our  visual  designer  to  give  it  a  professional  look  and  feel. 

Consistency  was  further  maintained  through  examination  of  overlapping  areas  between 
different  categories.  The  author  for  a  given  section  served  as  a  reviewer  for  the  other  sections. 
As  such,  each  author  was  able  to  identify  areas  where  another  author  was  describing  similar 
material  and  could  come  to  an  agreement  as  to  how  that  material  could  be  presented  in  a 
consistent  fashion.  In  some  instances,  the  solution  was  to  move  a  topic  from  one  category  into  a 
different  category  where  it  would  make  better  sense.  In  other  instances,  fee  same  material  was 
duplicated  and  presented  in  multiple  categories.  And  in  additional  instances,  similar  material 
was  presented  from  different  perspective  appropriate  for  fee  category  in  which  it  was  presented 
(e.g.,  email  systems  were  described  at  different  levels  of  granularity  for  fee  Electronic 
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Communications  category  and  the  Applications  category).  Overall,  as  indicated  by  subject 
matter  experts  (SMEs)  from  the  computer  engineering/programming,  professional/academic 
♦raining,  and  military  operations/training  fields,  the  authoring  and  review  process  allowed  the 
♦pam  to  create  consistent,  accurate  content  at  an  appropriate  level  of  detail  to  train  novice  users  in 
the  basic  computer  skills  categories. 

Questions 

The  questions  used  to  assess  the  trainee’s  knowledge  level  were  developed  directly  from 
the  reviewed  and  approved  concept  instructions.  If  the  student  answers  a  question  wrong  and  is 
consequently  sent  back  to  the  instructions,  he  will  be  able  to  find  the  answer  to  the  question 
within  these  instructions.  Questions  are  either  True/False  or  multiple  choice  with  a  minimum  of 
three  choices.  After  each  question  is  answered,  immediate  feedback  lets  the  trainee  know  if  his 
answer  is  right  or  wrong.  When  the  answer  is  correct,  the  trainee  receives  a  confirmation  that  he 
answered  the  question  correctly  along  with  the  correct  statement,  occasionally  with  further 
information  regarding  the  question  content.  In  this  way  the  correct  information  is  reinforced. 
When  the  answer  is  wrong,  the  trainee  receives  feedback  that  the  answer  was  incorrect  and  why. 
To  determine  the  correct  answer,  however,  he  must  read  the  concept  instructions.  An  example  of 
a  question  with  feedback  can  be  seen  in  Table  1. 


Table  1 .  A  Question,  Answer  C 

tioices,  and  Feedback  Found  in  the  Database. 

Question 

Choices 

Feedback 

instructions. 

Correct!  A  thread  is  a  part  of 
a  program,  or  list  of 
instructions,  that  can  execute 
independently  of  other  parts  of 
the  program. 

processes. 

Incorrect,  a  thread  is  not  made 
up  of  processes. 

A  thread  is  made  up  of: 

programs. 

Incorrect,  a  thread  is  not  made 
up  of  programs. 

the  concept  title.  For  example,  in  a  concept  called  Threads,  the  trainee  would  never  be  presented 

a  question  like  the  following:  “A _ is  made  up  of  instructions”,  where  Thread  is  the 

correct  answer.  Choices  such  as  “all  of  the  above”  and  “none  of  the  above”  were  also  avoided 
because  the  tool  will  present  answer  choices  to  the  trainee  in  a  random  order.  Moreover,  “all  of 
the  above”  and  “none  of  the  above”  tend  to  make  the  question  scoring  more  difficult  to  justify, 
because  test-takers  can  often  legitimately  argue  for  such  an  answer  despite  the  test-writer’s 
intentions.  In  computer-based  testing,  it  is  particularly  important  to  avoid  any  potential 
ambiguity  in  items,  since  there  may  be  no  available  human  resource  for  the  user  to  turn  to  with 
concerns. 


Bridge 

The  next  fundamental  area  for  content  development  was  the  “bridge”  portion  of  the 
program.  By  design,  this  content  is  totally  dependent  on  the  system  for  which  the  LSB  program 
is  being  used  (e.g.,  AFATDS).  To  develop  this  material,  subject  matter  experts  (SMEs)  defined 
the  types  of  material  trainees  should  be  presented  for  each  of  the  relevant  topic  areas. 
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Specifically,  for  the  AFATDS  program,  we  asked  AFATDS  instructors  at  FT  Sill  to  indicate 
which  topics  within  each  category  should  be  presented  to  the  trainee  (e.g.,  when  considering  the 
configuration  of  the  AFATDS  system,  is  it  more  like  a  star,  bus,  or  ring  computer  network  and 
how).  The  instructors  were  also  asked  specifically  how  the  basic  computer  concepts  available 
apply  to  AFATDS.  If  a  topic  was  not  relevant  in  regards  to  the  AFATDS  program,  it  was  not 
presented  to  the  trainee.  AFATDS  instructors  had  no  difficulty  deciding  which  content  was 
relevant  to  AFATDS.  Moreover,  the  instructors  took  some  time  to  explain  the  AFATDS  system 
in  detail  to  enable  the  inclusion  of  AFATDS  specific  diagrams  based  on  instructor  input. 

We  also  used  this  opportunity  to  have  the  instructors  tell  us  how  they  actually  use 
AFATDS.  This  information  was  crucial  for  the  “category  bridge”  that  summarized  how  each 
category,  in  its  entirety  (e.g.,  computer  networks),  is  operationalized  with  the  AFATDS  system. 
This  category  bridge  content  is  presented  after  the  trainee  has  successfully  completed  a  category. 
The  bridge  content  is  presented  to  help  the  trainee  have  a  better  understanding  of  the  “big 
picture”  and  how  these  skills  tested  are  applied  within  their  operations. 

The  last  sections  of  the  program  requiring  content  were  the  main  introduction  to  the  LSB 
program  and  a  general  description  of  the  specific  system  described  in  the  bridge  material.  The 
main  introduction  informs  the  trainee  why  they  are  using  this  program  and  gives  them  some 
background  on  the  system  for  which  they  are  being  trained.  Tlhe  instructions  explain  to  the 
trainee  how  to  navigate  within  the  LSB  program  and  the  fundamental  pages  that  will  be 
presented:  Assessment  Pages,  Instruction  Pages,  Bridge  Pages  and  My  Progress  Page. 

The  Bridge  “Philosophy.  ”  The  Bridge  is  central  to  the  LSB  program,  and  makes  the 
program  different  from  a  standard  program  that  teaches  computer  skills.  The  bridge  uses  digital 
concepts  that  have  just  been  learned  and/or  reviewed  by  the  user  and  maps  those  concepts  onto 
the  topic  of  interest,  in  this  case  AFATDS.  Thus,  immediately  after  learning  and/or  reviewing 
the  general  digital  topic  “Types  of  computer  networks,”  the  user  learns  how  AFATDS  is  a 
network.  After  learning  and/or  reviewing  information  on  ‘Types  of  user  interfaces,”  the  user 
learns  about  the  user  interfaces  in  AFATDS.  Of  course,  the  Bridge  content  can  be  modified  to 
address  any  digital  platform  or  technology.  This  makes  the  LSB  program  highly  adaptable  to  the 
needs  of  the  US  Army. 

We  believe  that  the  combination  of  questions,  instructions  and  remediation  (on  the 
general  digital  skills  content),  combined  with  the  Bridge  (general  content  applied  to  specific 
digital  platform)  represents  the  best  practices  in  pedagogic  design.  For  example,  one  of  the  facts 
haunting  instruction  theory  and  practice  is  that  generalizability  of  even  well-learned  principles 
and  facts  is  limited  (Alliger,  Tannenbaum,  &  Bennett,  1995).  By  immediately  instantiating 
learned  general  digital  skill  knowledge  into  the  context  of  a  particular  tool  or  platform,  however, 
the  learner  is  able  to  “make  the  connection”  between  concrete  and  particular.  In  effect,  the 
generalization  process  is  being  assisted.  “Network  configuration”  is  no  longer  just  a  concept,  but 
is  particularized  as  a  specific  kind  of  network  for  a  particular  tool  or  platform. 

Beyond  this,  the  Bridge  has  the  additional  advantage  of  making  die  general  into  the 
particular  in  an  entirely  different  way  -  through  personalization  of  the  content.  General 
knowledge  is  just  that,  general,  and  not  especially  related  to  any  given  individual.  The  users  of 
the  LSB  program,  however,  will  be  shown  Bridge  material  that  relates  specifically  to  a  platform 
on  which  they  will  be  shortly  trained,  and  which,  in  fact,  may  be  central  to  their  US  Army  career 
identity  (e.g.,  their  Military  Occupational  Specialty;  MOS).  For  this  reason  they  can  be  expected 
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to  see  the  material  as  relevant  to  themselves  as  individuals.  Although  it  is  at  this  point  only  an 
inference,  the  following  argument  seems  reasonable:  making  general  information  personalized 
by  tying  it  to  a  platform  on  which  soldiers  are  to  be  trained  may  increase  motivation  to  learn  in 
that  training.  The  fact  that  the  platform  is  not  a  “black  box”  but  has  been  contextualized  through 
the  LSB  (e.g.,  AFATDS  is  not  just  a  computerized  system  of  unknown  character,  but  a 
networked  system  already  pre-framed  and  understood  as  possessing  many  characteristics  of  other 
networked  systems)  may  lead  to  a  greater  sense  of  willingness  to  tackle  the  formal  training 
program.  This  is  an  hypothesis  only,  of  course,  but  one  that  could  be  tested  without  difficulty, 
provided  the  right  participants  and  situation. 

Usability  Evaluation 

One  of  Aptima’s  in-house  usability  experts,  Dr.  Gabriel  Spitz,  performed  a  full  usability 
evaluation  of  the  LSB  site,  both  the  user  side  and  the  local  administrator  side.  Summaries  of  Dr. 
Spitz’  findings  and  the  actions  taken  in  response  are  detailed  below. 

User  Side 

The  objective  of  this  effort  was  to  perform  a  heuristic  evaluation  of  the  Learning  Skills 
Bridge  web  site  and  application.  This  evaluation  consisted  of  a  systematic  walk  through  the  site 
and  assessment  of  the  usability  aspects  of  each  page  type,  the  interaction  within  a  page,  and  the 
navigation  between  pages.  The  primary  usability  aspects  that  were  reviewed  and  evaluated 
during  this  effort  included  navigation,  functionality,  user  control,  feedback,  and  consistency. 

Overall  the  site  design  was  nice  and  appealing.  Although  this  was  not  part  of  the 
usability  evaluation,  the  content  of  the  site  was  well  thought  out,  the  language  used  was  clear  and 
straight  forward,  and  the  questions  and  the  options  provided  were  very  well  constructed.  The 
graphics  and  color  schema  of  the  site  were  also  very  well  thought  out  and  presented  in  an 
effective  and  aesthetically  pleasing  way.  The  area  where  the  site  or  module  needed  to  be 
improved  was  in  its  usability.  At  a  high  level  of  abstraction  some  of  the  usability  issues 
identified  include: 

•  Unclear  conceptual  model  of  the  module  due  to  absence  of  a  site  and  module  home 
pages 

•  Several  instances  of  inconsistency  with  user  interface  design  conventions  such  as  using 
the  hand  pointer  when  displaying  “What’s  This”  information 

•  Missing  functionality  such  as  a  mechanism  to  deal  with  forgotten  passwords. 

From  these  recommendations,  improvements  were  made  to  the  login  page  (e.g.,  “forgot 
your  password?”  link)  and  the  introduction  page  (more  information  about  the  target  system  - 
AFATDS  -  and  how  the  LSB  program  will  help  the  user  better  learn  the  target  system  were 
included).  “Help”  pages  and  a  “log-out”  page  were  added.  The  look  and  feel  of  the  “my 
progress”  page  was  changed  to  allow  the  student  to  more  quickly  see  what  they  have  left  to  do  to 
complete  the  training  program. 


Local  Administration 

Dr.  Spitz  also  performed  a  similar  evaluation  for  the  local  administrator  pages.  Again, 
his  expert  advice  was  taken  and  changes  in  the  interface  were  implemented.  These  changes 
include  making  the  login  page  more  visually  appealing  and  adding  a  “forgot  your  password?” 
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link;  better  navigation  by  allowing  the  local  administrator  to  access  all  three  major  sections  from 
any  page  within  the  site;  adding  a  log-out  capability;  improving  the  student  reports  by  allowing 
the  local  administrator  to  sort  and  select  which  to  view;  and  improving  the  content  management 
capabilities  of  the  program. 

Validation  Research 

Initial  Data  Collected  with  National  Guard 

The  LSB  was  launched  with  the  National  Guard  Soldiers  of  the  147th  FA  Brigade,  1st 
BTN.  Data  were  collected  at  Camp  Ripley  to  determine  the  program’s  usefulness  and  face 
validity.  The  results  are  summarized  below. 

Participants  and  Methods 

Nine  National  Guard  Soldiers  (all  men)  participated  in  the  LSB  training  program  before 
participating  in  the  AFATDS  training.  They  were  given  instructions  for  the  LSB  program,  along 
with  IDs  and  passwords,  two  weeks  before  their  scheduled  AFATDS  training.  After  their 
AFATDS  training,  they  were  asked  to  give  feedback  on  the  utility  of  the  LSB  training  program 
in  preparing  them  for  the  AFATDS  training. 

Summary  of  Findings 

Overall,  the  Learning  Skills  Bridge  was  well  received  by  almost  all  of  the  individuals 
who  logged  on  and  used  the  system.  The  Networking  material  was  most  commonly  mentioned 
as  a  topic  that  provided  a  solid  foundation  in  terminology  and  understanding  prior  to  AFATDS 
training.  Most  interviewees  also  felt  that  LSB  could  be  useful  as  a  refresher  course  after  the 
completion  of  AFATDS  training  in  addition  to  its  current  pre-training  implementation.  More 
specific  to  the  method  of  implementation  in  this  program,  most  participants  had  very  positive 
comments  about  the  “answer  questions  to  determine  where  I  need  to  read”  method  of 
presentation  rather  than  being  required  to  go  through  the  entire  volume  of  text.  The  pictures  and 
diagrams  were  also  well  received  and  discussed  as  being  beneficial  for  understanding  concepts. 
Overall,  participants  felt  that  LSB  was  a  good  refresher  in  computer-related  concepts,  and  most 
thought  they  learned  something  they  didn’t  know  before  by  going  through  the  program. 

The  most  common  negative  comments  related  to  the  initial  learning  of  the  navigation 
scheme,  length  of  the  program,  and  a  desire  for  more  AFATDS-speeific  material.  Several 
participants  discussed  difficulties  learning  the  navigation  scheme  initially.  Comments  indicated 
that  there  was  no  clear  place  to  start.  Several  individuals  felt  there  was  too  much  need  for 
navigation;  they  would  prefer  a  system  that  required  fewer  decisions  and  simply  allowed  them  to 
walk  down  a  pre-set  path  to  completion  of  the  program.  Regarding  the  length  of  the  program, 
most  participants  felt  that  the  progress  pages  were  useful;  however,  several  indicated  that  they 
did  not  have  a  clear  idea  of  how  much  more  training  was  still  required.  Opinions  were  split  as  to 
whether  the  training  was  too  long  or  just  right,  with  a  slight  lean  towards  the  training  being  too 
long.  The  amount  of  time  required  for  completion  varied  greatly  between  participants  but  did 
not  seem  to  affect  their  opinions  as  to  whether  the  program  was  too  long  or  not  (i.e.,  some  of  the 
individuals  that  spent  the  most  time  in  the  program  felt  the  training  was  just  right).  Finally, 
interviewees  regularly  commented  that  they  expected  to  see  more  AFATDS-specific  information 
and  thought  that  tying  the  LSB  even  more  closely  to  the  AFATDS  training  would  be  beneficial. 
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Conclusion 

In  summary,  the  comments  and  suggestions  made  by  the  National  Guard  Soldiers  were 
invaluable  for  the  further  development  of  the  LSB  program.  For  a  more  rigorous  testing  of  the 
program’s  validity,  more  data  were  collected  at  FT  Sill,  OK,  on  September  23-26, 2002. 

Fort  Sill  Data  Collection 


Participants 

The  62  male  soldiers  had  a  mean  age  of  19.74;  64.5  percent  were  under  the  age  of  20;  the 
youngest  age  was  1 8,  the  oldest  30.  Most  of  the  soldiers  in  the  sample  were  privates:  43.5%  had 
a  rank  of  E-l,  40.3%  a  rank  of  E-2, 14.5%  had  a  rank  of  E-3;  there  was  a  single  E-4  (Specialist) 
in  the  sample.  The  MOS  of  all  soldiers  in  the  sample  was  13D,  Field  Artillery  Tactical  Data 
Systems  Specialist. 


Research  Design 


The  soldiers  were  given  pre-tests  in  two  LSB  AFATDS  modules,  Computer  Networks 
and  Visualization/Mapping.  The  soldiers  next  completed  these  two  modules  in  LSB  and  then 
completed  post-tests.  A  schematic  of  the  evaluation  design  is  shown  in  Figure  3.  While  the 
researchers  believe  that  the  results  of  this  study  are  sufficiently  strong  to  mitigate  the  threat  to 
validity  associated  with  quasi-experimental  designs,  the  accuracy  of  conclusions  regarding  the 
impact  of  training  on  pre-  and  post-test  differences  may  be  questioned  without  the  use  of  a 
control  group. 


Pre-tests 
mpleted 


Figure  3.  Research  Design  for  Evaluation  at  Fort  Sill 


Materials  and  Procedure 

The  testing  was  conducted  with  small  groups  of  up  to  16  people.  Each  room  had  16 
computers,  all  with  Internet  accessibility.  The  participants  were  provided  a  quick  rationale  for 
their  participation,  and  then  asked  to  complete  a  pre-test.  After  the  pre-test  was  collected,  they 
were  instructed  to  logon  to  the  LSB  and  complete  two  modules:  Visualization/Mapping  and 
Computer  Networks.  These  two  were  chosen  for  the  following  reasons:  AFATDS  is  a  highly 
networked  system,  and  maps  are  a  crucial  AFATDS  application.  After  all  individuals  had 
completed  the  LSB  modules,  they  were  asked  to  complete  the  post-tests.  They  could  not  look 
back  at  any  LSB  screens  during  the  post-test. 

The  Computer  Networks  test  was  a  20-item  test,  the  Visualization/Mapping  test  had  10 
items.  The  tests  were  True/False/Don’t  Know  format.  Because  these  tests  contain  sensitive 
information  concerning  the  AFATDS  training  program,  they  are  not  included  in  this  report. 


Sample  Description 

Participants  reported  varying  levels  of  computer  experience,  as  seen  in  Figure  4.  The 
anchors  for  this  question  had  the  following  descriptors:  Slight  =  e.g.,  have  spent  some  time  with 
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computer  games  or  surfing  the  web;  Basic  =  e.g.,  some  word  processing,  email,  surfing  the  web; 
Intermediate  =  e.g.,  word  processing,  email,  spreadsheet,  surfing  the  web;  Advanced  =  e.g.,  able 
to  troubleshoot  computer  problems,  programming.  One  reason  to  collect  this  information  was  to 
be  able  to  examine  an  hypothesis  of  interest:  the  greater  the  prior  computer  experience,  the  better 
participants  would  do  on  the  pre-tests,  while  still  showing  an  improvement  on  the  post-tests. 


Figure  4.  Number  of  Participants  Indicating  Varying  Levels  of  Computer  Experience 


Participants  were  also  asked  whether  they  were  familiar  with  AFATDS.  46.8%  said 
“No,”  while  51.6%  said  “Yes.”  One  participant  did  not  answer  this  question.  Of  those  who  said 
they  were  familiar  with  AFATDS,  most  reported  some  degree  of  training  experience,  from  1  to 
several  weeks.  This  question,  like  the  question  on  computer  experience,  was  asked  because  it 
was  thought  that  test  performance  might  differ  based  on  the  answer  participants  provided. 

Test  Score  Results 

Participants  did  better  on  both  the  networks  and  the  visualization/mapping  tests  after  LSB 
training  than  they  did  before.  The  descriptive  statistics  for  the  tests  are  shown  in  Table  2.  Note 
the  change  in  means,  pre  to  post,  and  also  the  increase  in  the  “floor”  or  minimum  scores.  The 
internal  consistency  of  the  tests,  as  indexed  by  Coefficient  Alpha,  was  acceptable  (the  lower 
alpha  for  the  post-test  Visualization/Mapping  test  is  probably  simply  indicative  of  a  multi-factor 
test). 
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Table  2.  Descriptive  Statistics  for  Pre-  and  Post-Test  Scores 


significance”).  Figure  7  shows  pre  and  post  means,  with  the  standard  error  of  the  means  plotted 
as  error  bars.  Two  conclusions  can  be  made:  first,  that  participants  found  the  test  on 
visualization  and  mapping  substantially  easier  than  the  test  on  computer  networks,  and  second, 
that  the  performance  on  the  post-tests  was  substantially  better  than  performance  on  the  pre-tests. 
That  is,  a  real  gain  occurred  in  knowledge  (as  measured  by  the  tests)  -  the  soldiers  knew  more 
about  both  AFATDS  visualization/mapping  and  AFATDS  networking  in  the  post-test  than  they 
did  in  the  pre-test.  It  is  possible  that  the  higher  post-test  scores  resulted  from  the  practice  effect 
of  the  pre-test;  however,  the  findings  were  so  strong  that  there  seems  to  be  a  main  effect  for  the 
LSB  training  such  that  the  training  resulted  in  the  increased  knowledge. 


Figure  7.  Mean  Pre-  and  Post-test  Scores 

Comparing  pre-  and  post-test  scores  for  soldiers  with  some  versus  no  AFA  TDS  training. 
One  hypothesis  was  that  soldiers  with  some  AFATDS  experience  would  do  better  on  the  pre¬ 
tests  and  on  the  post-tests  than  those  without  AFATDS  experience.  This  hypothesis  held  true  for 
the  pre-tests;  however,  LSB  training  seemed  to  eliminate  any  post-test  differences  due  to  prior 
AFATDS  experience  training  for  Visualization/Mapping  (see  figure  8).  Note  that  the  quasi- 
experimental  nature  of  this  study  makes  it  impossible  to  state  with  certainty  whether  these 
findings  were  the  result  of  learning  from  the  training  or  rather  the  result  of  having  experienced 
the  pre-test  (Cook  &  Campbell,  1979).  Such  a  “testing  effect”  would  result  in  increased  post-test 
performance  for  all  participants;  however,  it  would  not  be  expected  to  result  in  the  elimination  of 
differences  due  to  prior  experience  that  was  found  in  this  study. 
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Figure  8.  Mean  Pre-  and  Post-test  Scores  by  Prior  AFATDS  Training 
Comparing  pre-  and  post-test  scores  by  level  of  reported  computer  experience.  Prior 
computer  experience  correlated  with  Networks  test  performance,  as  shown  in  Table  3.  This  is 
reasonable,  given  that  computer  networks  is  more  likely  than  mapping  to  be  understood  by 
someone  with  some  computer  experience. 


PREMAP 

POSTMAP 

PRENET 

POSTNET 

r 

.021 

.171 

Km 

.368 

2-tailed  sig 

.869 

.189 

Km 

.004 

N 

61 

61 

61 

61 

When  the  mean  scores  are  broken  out  by  level  of  computer  experience,  this  correlation 
between  computer  experience  and  networks  test  scores  is  seen  clearly  (see  Figure  9).  Moreover, 
the  upward  trend  in  means  on  the  networks  test  across  levels  of  computer  experience  is  evident 
in  the  post-test  as  well  as  the  pre-test. 
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Figure  9.  Mean  Pre-  and  Post-test  Network  Scores  by  Computer  Experience 


Comparing  pre-  and  post-test  scores  by  both  level  of  reported  computer  experience  and 
prior  AFATDS  training.  Figures  10  and  11  show  the  influence  of  both  computer  experience  and 
prior  training  on  test  performance.  In  general,  the  graphs  can  be  interpreted  as  showing  that 
computer  experience  assisted  soldiers  on  the  Networks  test  only,  while  AFATDS  experience 
assisted  soldiers  in  both  Networks  and  Visualization/Mapping.  This  makes  sense,  since  mapping 
is  a  topic  that  should  not  be  assisted  by  prior  computer  knowledge,  while  prior  AFATDS 
experience  should  assist  participants  in  both  the  mapping  and  networks  areas. 
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Figure  10.  Mean  Pre-  and  Post-test  Visualization  Mapping  Scores  by  Prior  AFATDS  Training 

and  Computer  Experience 


Figure  11.  Mean  and  Pre-  and  Post-test  Computer  Network  Scores  by  Prior  AFATDS  Training 

and  Computer  Experience 

Participant  Reactions.  Our  observations  during  the  training  sessions  indicated  that 
participants  were  able  easily  to  complete  the  LSB  modules.  Moreover,  a  reactions  questionnaire 
was  administered.  The  responses  to  some  of  these  questions  were  content  coded;  the  results  of 
this  content-coding  are  shown  in  Table  4.  These  responses,  as  a  whole,  show  a  very  favorable 
response  to  different  aspects  of  the  LSB  experience. 


Table  4.  Responses  to  the  Reactions  Questionnaire 


Did  you  find  the  introduction  to  be  effective  at  explaining 
the  purpose  of  the  system  and  how  to  use  it? 


2.  Was  the  system  easy  to  navigate  through? 


3.  Were  the  assessment  questions  too  easy?  Too  difficult?  Just 
right? 


4.  Were  the  lessons  written  in  a  way  that  made  the  concepts 
easy  to  understand? 

5.  Do  you  feel  that  the  system  improved  your  understanding  of 
AFATDS? 


Yes -90.3% 
No-  3.2% 


Somewhat  -  4.8% 

_ No  response  -  1.6% 

Yes -98.4% 

_ No  response  - 1.6% 

Too  easy  - 11.1% 
Too  difficult  -  6.3% 
Just  right -71.4% 
Other  response  -  9.5% 
Yes -72.5% 
No -4.8% 
Somewhat  or  Other  -  22.7% 
Yes,  considerably -8.1% 
Yes,  slightly  --19.4% 
Yes  -  61.3% 
No— 9.7% 


The  participants  also  indicated  what  they  liked  and  did  not  like  about  the  system.  Most 
responded  that  the  system  was  informative  and  easy  to  understand.  Many  said  that  it  was 
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instrumental  in  teaching  them  about  hardware  and  software  and  the  AFATDS  program.  A 
majority  of  the  respondents  could  not  think  of  anything  about  the  LSB  that  needed  improvement, 
or  their  suggestions  were  superficial  (e.g.,  “color  scheme”);  however,  eight  soldiers  thought  the 
program  was  too  long  or  had  too  many  questions. 

Research  Conclusions  and  Implications. 

In  summary,  die  LSB  program  was  very  well  received  by  the  Soldiers.  They  responded 
positively  to  it,  generally  finding  it  easy  to  use  and  believing  it  to  be  helpful  in  preparing  them 
for  AFATDS  training.  Testing  results  showed  significantly  improved  scores  on  the  AFATDS 
material  and  knowledge  of  basic  computer  systems. 

The  effects  of  prior  computer  experience  and  prior  AFATDS  training  affected  LSB 
performance  as  expected:  relevant  test  scores  were  improved;  however,  it  is  useful  to  point  out 
that  neither  of  these  variables  eliminated  the  much  more  powerful  main  pre-post  effects  for  both 
of  the  examined  AFATDS  modules.  The  primary  lesson  here  is  that  the  LSB  seems  to  be  a  potent 
method  of  learning  that  can  be  of  benefit  even  to  users  with  some  prior  computer  experience 
and/or  some  exposure  to  AFATDS. 

It  should  be  remembered  that  this  research  examined  performance  on  only  two  of  the 
LSB  AFATDS  modules.  One  reasonable  question  is  whether  the  increases  in  performance  that 
were  shown  for  both  of  the  modules  in  this  research  (Computer  Networks, 
Visualization/Mapping)  also  will  occur  on  the  larger  scale  of  all  the  modules  that  apply  to 
AFATDS.  This  question  can  only  be  definitively  answered  empirically.  However,  likely  gains 
from  pre-  to  post-test  would  be  similar  in  all  AFATDS  modules  if  we  make  two  assumptions:  1) 
assume  a  matrix  of  generally  positive  intercorrelations,  as  found  previously  among  tests  of 
achievement  (Alliger,  1988);  and  2)  further  assume  that  the  learning  that  can  be  gained  from  the 
other  LSB  modules  is  of  similar  magnitude  since  each  content  area  was  written  according  to  the 
same  standards  and  subject  to  the  same  review. 

If  there  was  a  perceived  negative  to  the  LSB  system,  it  is  that  to  some  soldiers  the 
experience  seemed  “too  long”  or  “too  drawn  out.”  This  probably  is  due  in  part  to  the  fact  that  the 
LSB  is  academic,  requiring  users  to  study  material  and  answer  questions,  understand  remediation 
-  in  short,  to  be  a  student.  Being  a  student  requires  attentiveness,  which  can  be  tiring  over  a 
period  of  time.  Given  that  this  perception  was  reported  by  a  small  minority  of  soldiers,  it  is  not  in 
itself  excessively  worrisome.  On  the  other  hand,  in  this  research  soldiers  were  exposed  to  only 
two  modules.  In  a  complete  rollout  of  the  system,  the  perceptions  of  tedium  could  increase  since 
there  would  be  substantially  more  material  to  cover.  This  suggests  that  the  “Start/Stop/Restart” 
feature  of  the  program  is  important.  That  is,  because  users  can  begin  an  LSB  session,  stop  when 
necessary,  and  resume  when  convenient  without  losing  any  progress  through  the  program,  issues 
of  boredom  and  fatigue  are  minimized.  Add  to  this  the  fact  that  learning  seems  best  when  it  is 
distributed  in  sessions  over  time  (Baddeley  &  Longman,  1966)  Probably  the  recommended  way 
for  users  to  experience  the  LSB  is  in  a  time-distributed  manner. 

Overall,  then,  the  Ft.  Sill  research  effort  indicates  that  the  LSB  can  provide  substantial 
learning  to  candidates  for  AFATDS  training.  Further  research  could  extend  the  current  findings 
in  a  number  of  ways,  however,  and  is  recommended.  Of  particular  interest  are  the  effects  of  full 
LSB  training  on  actual  performance  in  AFATDS  and  on  field  performance.  Also,  experimental 
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studies  incorporating  a  control  group  should  be  performed  to  fully  determine  the  effectiveness  of 
the  LSB  training,  and  it  would  also  be  useful  to  assess  group  differences  based  on  ethnicity. 

Final  Product 

Based  on  the  feedback  from  the  usability  studies  and  data  collection,  improvements  were 
made  to  the  LSB  program.  The  final  product  for  this  project  is  a  dynamic  web-based  training 
program  (Learning  Skills  Bridge).  It  is  designed  to  both  enhance  current  and  future  technology- 
based  training  and  to  enhance  transfer  of  learned  skills  to  the  field  setting.  For  those  who  do  not 
have  the  opportunity  to  experience  this  program  first  hand,  we  have  provided  a  walk  through  of 
the  program  in  the  Appendix. 

We  believe  that  a  clear  need  is  met  by  the  Learning  Skills  Bridge,  that  it  possesses  a 
compelling  and  pedagogically  superior  design,  and  that  results  to  date  indicate  its  potential.  The 
LSB  is  designed  to  allow  local  administrators  extensive  control  over  how  the  program  uses  its 
core  information  to  tie  into  a  target  field  application,  tool,  or  platform.  Users,  too,  have 
substantial  freedom  in  navigation  and  in  scheduling  their  participation.  In  short,  we  believe  that 
the  LSB  has  a  successful  and  highly  adaptable  design,  and  will  meet  the  need  for  military  and 
non-military  users  and  administrators  alike  in  training  adaptability  in  digital  skills. 

Overall  Conclusions  and  Implications 

This  SBIR  Phase  II  has  not  only  provided  a  great  deal  of  evidence  indicating  the  need  for 
digital  skills  -  as  revealed  by  participant’s  reactions  and  comments  during  our  final  data 
collection  -  but  it  has  provided  evidence  for  the  usefulness  of  training  general  computer 
knowledge  to  achieve  adaptability  in  digital  skills.  The  great  majority  of  participants  reported 
benefiting  greatly  from  this  training  and  felt  the  training  better  prepared  them  for  using  the  target 
system  (i.e.,  AFATDS). 

While  our  training  philosophy  and  methodology  is  more  than  sufficient  to  result  in  digital 
skill  adaptability,  we  cannot  say  that  it  is  necessary  for  such  adaptability.  In  other  words,  it  is 
clear  that  the  combination  of  principle  and  identical  elements  theories,  as  demonstrated  in  the 
LSB  training,  results  in  adaptability  of  digital  skills,  but  we  can  not  say  that  this  result  is  better 
than  the  application  of  either  of  the  theories  alone.  To  remedy  this,  we  recommend  a  trial 
implementation  of  the  LSB  in  the  AFATDS  environment,  with  a  long-term  plan  to  follow 
trainees  through  AFATDS  training  and  out  into  the  field.  Such  an  implementation  could 
continue  to  test  the  effectiveness  of  the  LSB  in  one  targeted  training  environment,  and  the 
longitudinal  nature  of  such  a  study  -  which  would  last  through  introductions  of  new  versions  of 
the  AFATDS  computer  system  -  would  allow  for  further  and  more  rigorous  testing  of  the  LSB’s 
ability  to  train  adaptability  in  digital  skills. 

In  addition,  it  would  seem  important  to  adapt  the  LSB  (and,  as  mentioned,  ease  of 
adaptation  was  “designed  into”  the  system  from  earliest  specifications)  to  a  new  platform  besides 
AFATDS.  One  of  the  many  other  U.S.  Army  platforms  requiring  digital  skills  could  serve  as  a 
testbed  for  examining  the  effectiveness  of  the  LSB.  Such  ongoing  research  can  provide 
information  that  would  allow  the  LSB  to  be  as  effective  as  possible  in  promoting  digital  skills 
and  optimal  training  performance  in  the  US  Army. 

Commercialization 

In  addition  to  providing  a  compelling  product  for  training  adaptability  in  digital  skills  for 
AFATDS  operators/trainees,  the  LSB’s  design  affords  a  much  broader  application.  With  the  two- 
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module  design,  re-tailoring  the  bridge  module  to  a  new  domain  should  benefit  a  new  (i.e.,  non- 
AFATDS)  training  area.  As  such,  Aptima  has  begun  a  process  of  exploring  the  possibilities  by 
which  this  technology  can  be  applied  to  a  wide  variety  of  jobs  and  settings. 

As  a  first  step,  research  has  been,  and  continues  to  be,  conducted  to  assess  the  current 
market  state  for  web-based  asynchronous  learning  management  systems  in  order  to  identify  the 
market  potential  for  the  Learning  Skills  Bridge  (LSB)  product.  Specific  markets  Aptima  is 
actively  pursuing  outside  of  the  defense  sector  include  policing  agencies,  emergency 
management  agencies,  hospital  settings,  and  secondary  education  settings.  Aptima’s  reason  for 
targeting  these  markets  is  two  fold.  First,  these  settings  have  been  identified  as  supporting  and 
benefiting  from  technology  use  and  have  segments  of  their  population  who  are  novice 
technology  users.  Secondly,  these  are  areas  in  which  Aptima  has  a  high  level  of  generalizable 
domain  experience.  We  believe  that  the  combination  of  these  factors  combined  with  the  ease  by 
which  our  system  is  adaptable  provides  a  very  attractive  incentive  to  trainers  in  these  fields. 

Successful  commercialization  will  be  achieved  through  a  variety  of  methods  including 
professional  conference  presentations,  full-scale  validation  studies  in  field  settings,  and  direct 
marketing  through  partnering  with  organizations  who  have  a  niche  in  the  fields  of  interest.  It  is 
important  to  note  that  these  efforts  directed  to  the  private  sector  will  serve  to  compliment  our 
commercialization  efforts  in  the  govemment/defense  arenas.  Aptima  will  continue  to 
aggressively  market  our  technology  to  address  needs  both  within  the  Army  and  other  branches  of 
the  military,  as  well  as  other  government  organizations.  As  evidence  of  our  commitment  to  the 
goal  of  commercialization,  we  have  submitted  a  proposal  and  pricing  list  in  an  effort  to  be 
registered  under  the  General  Service  Administration’s  Management,  Organizational  and 
Business  Improvement  Services  (GSA  MOBIS)  schedule.  We  are  confident  that  our  efforts  will 
result  in  successful  commercialization  both  within  the  government  and  private  sectors. 
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Appendix.  Walk-Through  of  the  LSB  Learning  Accelerator  Training  Program 
This  Appendix  describes  the  product  from  a  user  standpoint.  It  first  walks  through  the 
introduction  section.  The  reader  will  then  be  guided  through  the  introduction  of  the  first  category 
-  computer  networks,  and  through  the  introduction  of  the  first  topic  within  this  category  -  types 
of  computer  networks.  The  walk-through  then  precedes  through  one  concept  for  that  topic,  in 
this  case  LAN.  In  addition,  the  Appendix  will  demonstrate  special  features  of  the  program 
including  “rollover”  capabilities,  instruction  pages,  progress  pages,  bridge  pages,  and  help  pages. 
Our  hopes  are  that  this  will  provide  the  reader  with  an  understanding  of  the  capabilities  and 
features  of  the  LSB  training  program. 


A  user  begins  by  typing  in  the  following  URL  into  his  Internet  browser, 
http ://www,leamin gskillsbridge.com/AFATDS/.  Once  on  the  AFATDS  main  page  (Figure  A-l), 
users  may  access  the  program  by  typing  in  their  user  id  and  password. 


Figure  A-l .  Screenshot  of  User  Login  Page 


For  those  who  have  forgotten  their  password,  a  link  is  available  labeled  “forgot 
password”.  Clicking  on  this  link  takes  the  user  to  a  page  in  which  they  are  prompted  to  enter 
their  user  ID,  password,  and  confirmation  of  password  (Figure  A-2).  Clicking  the  “submit” 
button  sends  an  email  to  the  global  administrator  prompting  the  global  administrator  to  verify  the 
validity  of  the  email  address  and  user.  Upon  verification,  the  global  administrator  will  then  email 
a  new  password  to  the  user  within  24  hours. 
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Figure  A-2.  Screenshot  of  “Forgot  Your  Password”  Page 
If  the  user  types  an  incorrect  user  id  and/or  password,  the  user  receives  the  message 
“illegal  login,  please  try  again”  beneath  the  login  text  box  (see  Figure  A-3).  Once  a  user  has 
successfully  logged  in,  they  are  taken  to  the  introduction  page  (Figure  A-4). 


Figure  A-3.  Screenshot  of  “Illegal  Login”  Page 


The  introduction  page  is  highlighted  by  a  variety  of  categories  running  horizontally 
across  the  top  margin  of  the  page  and  topics  running  vertically  down  the  left  hand  margin  of  the 
page.  The  topic  choices  (left  margin)  will  change  depending  on  which  category  (top  margin)  is 
highlighted. 
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Introduction  to  AFATDS 


Welcome!  This  program  is  designed  to  do  three  things: 

1 .  diagnose  your  knowledge  of  computer-related  information 

2.  provide  you  instruction  in  computer-related  topics  where  you  need  it,  and 

3.  show  you  how  this  information  relates  to  the  Advanced  Field  Artillery  Tactical  Data  System 
(AFATDS)  on  which  you  will  soon  be  trained. 

AFATDS  Overview 

AFATDS  is  a  fairly  complicated  system,  as  illustrated  in  the  figures  below.  This  program  is  designed 
to  ensure  that  you  have  the  background  knowledge  needed  to  get  the  most  out  of  the  AFATDS 
training  you  will  soon  be  taking. 

This  first  figure  shows  the  hardware  used  for  AFATDS. 
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Figure  A-4.  Screenshot  of  “Introduction  to  AFATDS”  Page 

Walking  through  the  introduction  topics  down  the  left  hand  side,  a  user  is  provided  with 
useful  information  regarding  the  purpose  of  the  program  (introduction),  description  of  the 
navigation  layout  for  the  LSB  program  (Figure  A-5),  descriptions  and  screenshots  of  assessment 
pages  (Figure  A-6),  an  explanation  of  how  instruction  pages  may  assist  users  (Figure  A-7), 
descriptions  of  the  bridge  components  (Figure  A-8),  and  information  regarding  how  users  may 
track  their  progress  through  the  program  (Figure  A-9). 
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The  next  several  pages  will  walk  you  through  the  navigation  scheme  and  the  program  flow.  If  you 
have  reached  this  page  because  you  are  experiencing  problems  on  a  specific  page  (i.e  ,  an 
assessment  page,  instruction  page,  bridge  page,  or  my  progress  page),  please  select  the  link  for 
help  on  that  page  on  the  green  navigation  bar  to  the  left. 


Navigation 


Navigation  through  the  Learning  Skills  Bridge  is  easy.  The  diagram  below  highlights  the  major 
menus. 
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Figure  A-5.  Screenshot  of  “Navigation”  Page 
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Assessment  Pages 

After  you  choose  a  topic,  you  proceed  through  the  "Answer  Questions,  Instruction  as  Needed,  and 
Bridge"  flow.  This  process  begins  with  questions  on  the  assessment  pages. 


Each  question  has  two  to  four  choices,  but  only  one  correct  answer.  If  you  get  a  question  correct, 
you  proceed  to  the  next.  If  you  get  a  question  wrong,  you  view  the  instructional  material,  then  go  to 
the  next  question. 


An  example  assessment  page  is  shown  below.  Note  the  diamonds  in  the  menu  on  the  left  -  when 
you  have  completed  a  section,  a  blue  diamond  will  appear  next  to  the  section  so  that  you  always 
know  what  you  have  completed  and  what  you  have  left  to  complete. 
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Figure  A-6.  Screenshot  of  “Assessment  Pages”  Page 
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Figure  A-7.  Screenshot  of  “Instruction  Pages”  Page 
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Bridge  Pages 

Alter  you  hare  completed  a  topic ,  you  can  view  the  Bridge  page  for  that  topic.  This  page  explains 
how  the  computer-related  knowledge  for  that  topic  relates  to  your  target  system. 

An  example  Bridge  page  is  shown  below. 
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Figure  A-8.  Screenshot  of  “Bridge  Pages”  Page 
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My  Progress  Page 

At  any  time,  you  can  click  on  “My  Progress"  in  the  left-hand  menu,  and  identify  how  many  questions 
you  have  answered  correctly  for  each  topic  and  the  number  you  are  still  required  to  answer  for  each 
topic.  “My  Progress"  is  a  kind  of  road-map  you  can  check  whenever  you  wonder  how  far  you  have 
come,  or  need  still  to  go,  in  the  Learning  Skills  Bridge. 

An  example  My  Progress  page  is  shown  below. 
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Figure  A-9.  Screenshot  of  “My  Progress  Page”  Page 


The  introduction,  as  with  the  rest  of  the  LSB  program,  is  designed  to  flow  from  one  topic 
to  the  next  through  the  use  of  a  “continue”  button  at  the  bottom  of  the  screen.  For  users  who 
prefer  to  focus  on  topics  in  a  different  order,  they  may  simply  click  on  the  appropriate  category 
and  topic  in  any  order  (see  Figure  A- 1 0). 


LEARNING  M 

iiiPWBlIiE 


IPItiptS 


>  My  progress 
Help 


Computer  j  Electronic 
Networks  {  Communications 


Components  Visualization/Mapping 


jpjjroiqi-ess: 

"At  anytime-;  y&itc$R  click  op  “My  Prioress- in  1he1e%hantf^emi^d  idt^pylhotff  maj^'qdestes: , 

i@ill  il  lit  Hi  ^'quire^to^rfs^tfdcWacK"^ 

^Progress"  is  a  kind  of  road-map  you  can  check  whenever  you 
came, S t  need  still  to  go,  m  the  teaming  Skills  Bridge 

\ M  example  My. Progress'  'page  Ts  shown’below,lC :  - *  *  M  §p>%  M  'V v  i  rt  -  '* 

: MS ' S ,.\V S- 


'H&v* 


LEARNING 

v  SKfUSBRIDGf:; 


nrwii?  $&<:  s  ^<K  t. :  '.:f  r 

■ '  ^  -v^  4>  . 


Ofii^S&sSiSS 

5  t ;  :' 

_‘V<  xd’  V' 


|C:iiri«rttS»rbgre«| 


4  ]Vv-'  wv-' „  .N;tl 

i 

SRiSllft^SI8|S 

’ 

-trt  $4  \  1-v  'i 


Student  Name:sarah2 

tV('Vr  c-  ’  I! ’</  ^ '”  W 

SS  i  $  m  >«7  ;S  >  ft  >  & 


fipii.T.jj  Inc.  -0!  U-3--.C 


Figure  A- 10.  Screenshot  pointing  out  ways  to  start  a  computer  skills  category 
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By  clicking  on  a  category,  in  this  case  “computer  networks,”  users  are  taken  to  an 
introduction  page  for  the  chosen  category  (see  Figure  A-ll).  The  introduction  sections  provide  a 
general  overview  of  the  selected  category.  Within  the  introduction,  key  or  important  terms  are 
bolded  and  are  active  (placing  cursor  over  bolded  text  provides  the  user  with  a  definition  of  the 
term  or  terms). 
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Introduction  to  Computer  Networks 


A  computer  network  is  defined  as  two  or  more  computers  (or  computerized  devices)  connected 
in  such  a  wav  that  informality  teuch  **  Start  f  en  he  and  the  computers  ten  communicate 
(in  acUj*d*vto§ tfwt  b  to pwfan crfaJattons and  for  «M<h  two  computers  or  computerized 

Jwont,  and  one  prodding  information, 

catted  the  server  (although  they  could  ham  equivalent  capabilities  pnd  responsibilities  in  a  peer-to- 
peer  network),  and  these  computerized  devices  must  have  a  connection  (which  may  be  wireless 
or  cable),  Computers  in  a  network  may  permit  people  to  communicate,  but  they  may  be  designed 
to  allow  dwkMorappfctiioneto  communicate,  that  is.  to  send  and  receive  information  among 
each  other.  However,  there  aie  usually  humans  changing  or  communicating  through  the  netwoft. 
even  if  the  network  is  mainly  set  up  to  let  machines  ’talk*  to  one  another. 

So.  networks  m  supposed  to  enable  Cast  and  convenient  sharing  of  information,  resources,  and 
capabilities  like  pricing  Productivity  increases  with  such  sharing,  for  exampte,  let's  say  your 
computer  cannot  handle  large  amounts  of  data  analysts,  You  can  use  a  network  to  borrow  or 
’share*  a  data  analysis  program.  The  network  would  allow  your  computer  to  send  the  data  to  the 
other  computer,  tell  ii  what  analyses  to  run,  and  receive  the  results  when  the  other  computer  is 
done. 


Figure  A-ll.  Screenshot  of  “Introduction  to  Computer  Networks”  (Category)  Page 


Clicking  continue  at  the  bottom  of  the  screen  takes  users  to  the  introduction  page  for  the 
first  topic  within  that  category  (see  Figure  A- 12),  in  this  case  “Types  of  Computer  Networks”. 

The  topic  introduction  pages  are  organized  similar  to  the  category  introduction  pages  (i.e. 
provide  general  overview  of  topic  and  bolded  terms  with  roll  over  capability  when  needed). 
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Introduction  to  Types  of  Computer  Networks 

A  network  can  be  large  and  serve  many  people  and  computers  or  small  and  serve  only  two  or 
three  people  and  computers.  Networks  may  also  serve  a  small  area  (as  in  a  Local  Area  Network) 
or  a  large  area  (as  in  a  Wide  Area  Network).  Here  we  review  Local  Area  Networks  (LANs),  Wide 
Area  Networks  (WANs),  and  the  intemet/intranet  distinction. 

To  start  answering  questions  within  this  topic ,  please  select  a  concept  to  the  left.  If  you  are  unsure 
of  the  answer,  you  may  select  the  "lnstnjctionM  button  to  read  more  about  the  concept. 
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Figure  A- 12.  Screenshot  of  “Introduction  to  Types  of  Computer  Networks”  (Topic)  Page 
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By  clicking  continue  at  the  bottom  of  the  topic  introduction  page,  users  are  taken  to  the 
assessment  pages  for  the  particular  concepts,  in  this  case  “LAN”  (see  Figures  A-13  and  A-14  for 
Assessment  Page  examples) 


The  assessment  pages  contain  questions  for  each  of  the  concepts  and  appear  in  a  random 
order.  Questions  take  the  format  of  true/false  (Figure  A-13)  or  multiple-choice  (with  at  least  3 
response  options;  Figure  A-14). 
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Figure  A-14.  Screenshot  of  Assessment  Page  with  Multiple  Choice  Question 
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Users  simply  click  the  radio  button  associated  with  their  choice  of  answers  to  the 
particular  question  and  then  click  the  “submit”  button.  If  die  user  does  not  choose  an  answer, 
they  are  taken  to  a  screen  indicating  that  they  did  not  do  so  (Figure  A-15),  and  by  clicking  the 
“continue”  button  are  taken  back  to  the  specific  question. 
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You  did  not  select  an  answer.  Click  the  Continue  button  to  return  to  the  assessment  page 


Figure  A-15.  Screenshot  for  When  a  Student  Does  Not  Select  an  Answer 

If  their  response  is  correct,  they  receive  confirmation  that  their  answer  is  correct  and  are 
prompted  to  click  continue  to  the  next  question  or  topic  (Figure  A-16). 


LEARNING 

SKU1SBR1DGE 


fnttoduclinn 


•  My  progress 

•  Help 

•  i-ogOut 


•  .’r  it'Uninic  ;  Components!  °|,e,dU,"J  T Vfeualimio.vMau,>i»«t I AppiicatioiTWr  ■ 

Cuiiiinunicdtions  <  j  Systems  i  1 1  J  |  Software  Interface  Bt- 

. :v "■ .  -  "  -  . ' - . 


Correct!  Every  computer  on  a  given  LAN  has  a  direct  connection  to  that  specific  LAN  and  its 
hardware. 


Figure  A-16.  Screenshot  of  a  “Correct  Response”  Feedback  Page 
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If  the  user’s  response  to  a  question  is  incorrect,  the  user  receives  a  message  indicating 
that  the  answer  is  incorrect  (Figure  A- 17). 


LEARNING 

SKILLSBRIDGE 


ssasffi 


•  My  progress 

•  Help 

•  Log  Out 


Figure  A-17.  Screenshot  for  a  “Incorrect  Response”  Feedback  Page 
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They  are  then  prompted  to  continue  on  to  the  instruction  pages  for  that  topic.  The 
instruction  pages  (See  Figure  A-18)  provide  important  information  for  the  current  concept,  in 
this  case  LAN,  which  may  include  definitions  and  information  of  how  this  concept  relates  to 
other  concepts  in  the  topic. 


mmm 


Iritfuduirtinn 


•  My  progress 

•  Help 

•  Log  Out 


j  FUictromc  ;  4  .  Operatum  w 

..  !  Components '  ~  J  t  V 

Lnmmumcdtitwis  i  Systems  | 


Operating  Tw.  ..  .  M  Apn hcdtiun  User 

Systems  |  ~  •;  Softwaie  ;  Interface  H 

. . ^  c-;  g 


A  LAN  is  a  network  that  is  limited  to  one  geographical  area  -  often  within  a  single  building  or  set  of 
buildings.  When  a  LAN  is  used  by  a  single  organization  or  group  of  people,  it  is  termed  an 
"Intranet "An  intranet,  like  an  intrastate  highway,  links  people  within  a  limited  area  (the  interstate 
crosses  state  borders,  but  an  intrastate  road  is  a  local  road  that  does  not  cross  state  borders).  An 
intranet  is  closed  off  to  users  outside  it  through  security  measures  like  a  "firewall"  or  pass  codes. 

A  LAN  of  computers,  or  network  nodes  are  formed  simply  by  direct  cable  (or  wireless)  connections 
between  at  least  two  computers.  A  LAN  can  be  as  small  as  two  computers,  or  as  large  as 
hundreds.  The  important  concept  is  that  every  computer  on  a  given  LAN  has  a  direct  connection  to 
that  specific  LAN  and  its  hardware  -  no  routing  or  indirect  connections  apply. 


Figure  A-18.  Screenshot  of  “LAN  Basic  Instructions”  Page 


It  is  important  to  note  that  users  may  access  the  instruction  page  for  the  concept  at 
anytime  during  the  assessment  process  by  simply  clicking  the  “instruction”  button  (Figure  A-19). 


Figure  A-19.  Screenshot  pointing  out  button  to  access  Instructions  from  the  Assessment  Pages 


BEST  AVAILABLE  COPY 


A-15 


best  available  copy 


After  answering  the  predetermined  number  of  question  for  a  concept  correctly,  the  user  is 
congratulated  (see  Figure  A-20)  and  then  taken  to  the  assessment  page  for  the  next  concept 
within  that  topic,  in  this  case  “WAN.” 
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Correct!  Just  as  within  a  LAN,  a  WAN  can  have  different  connection  types  and  methods. 
Congratulations!  You  have  completed  this  concept.  Click  continue  to  move  on  to  the  next  concept 
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Figure  A-20.  Screenshot  of  Page  Indicating  When  a  Concept  is  Finished 
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Once  the  predetermined  number  of  questions  for  all  the  concepts  for  a  topic  have  been 
answered  correctly,  users  are  taken  to  the  bridge  page  for  that  topic,  see  Figure  A-21  for  a 
sample  bridge  page.  The  bridge  pages  provide  an  explanation  of  how  the  current  topic  applies  to 
the  target  computer  system,  i.e.,  AFATDS  (for  export  control  purposes,  Figure  33  does  not 
contain  AFATDS-specific  information).  The  bridge  pages  often  include  both  pictorial 
representations  of  how  the  topics  relate  as  well  as  important  bolded  terms  with  rollover 
capability. 
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each  on  the  LAN,  b  bet,  e*?ry  node  on  the  network  has  an  iP  address.  So,  for 

example,  a  network  may  have  a  network-capable  printer  that  any  of  she  workstations  can  use  - 
such  a  printer  will  have  its  own  IP  address. 

IP  addressee  sometime  will  need  to  be  entered  ’into  a  diabj  box.  Remember  that  IP  addresses 
m  in  ihe  form  of  to  sets  of  numbers,  mt- h  as  t23.3123.34S.  So*  for  example,  yo$s  might  need  to 
mlm  in  an  IP  address  in  s  dialog  bmK  something  like  this: 


After  you  enter  ibe  correct  IP  address,  your  workstation  •will  be  able  to  “see*  the  olher  nodes 


Figure  A-2 1 .  Screenshot  of  “Bridge”  Page 
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This  general  process  continues  throughout  the  program.  After  finishing  all  of  the  topics 
within  a  category,  the  user  is  taken  to  a  “Category  Status”  page  (see  Figure  A-22),  that  provides 
information  regarding  the  number  of  correctly  answered  question  required  to  complete  a 
category. 
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Student:  saraM 

Congratulations  on  finishing  Operating  Systems.  Choose  one  | 
the  categories  you  have  not  yet  completed  to  continue.  The  table  1 
below  lists  the  categories  and  the  number  of  questions  in  each  that  1 
you  need  to  complete.  1 

Category 

Remaining  Questions  j 

Computer  Networks 

58  j 

Electronic  Communications 

16  .  1 

Components 

_  .  .  40  i 

Operating  Systems 

0  I 

Visualization/Mapping 

34  ! 

Application  Software 

8  | 

User  Interface 

32 

Figure  A-22.  Screenshot  of  “Category  Status”  Page. 
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It  is  important  to  note  that  users  may  access  their  “My  Progress”  page  (Figure  A-23)  at 
anytime  by  clicking  the  “my  progress”  links  located  in  the  upper  right  portion  of  the  screen  and 
the  bottom  of  the  left  vertical  navigation  bar. 
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Figure  A-23.  Screenshot  of  “My  Progress”  Page  pointing  out  also  how  to  get  there 


Another  feature  included  within  this  program  is  the  “help”  option.  By  clicking  either  of 
the  two  “help”  links  located  just  below  the  “my  progress”  links,  users  are  taken  to  the  navigation 
page  (See  Figure  A-5),  which  provides  users  with  assistance  related  to  navigation  of  the  site 
including  program  flow,  assessment  pages,  instruction  pages,  bridge  pages,  and  my  progress 
page. 


A-19 


BEST  AVAILABLE  COPY 


As  mentioned  earlier,  navigation  through  the  program  is  designed  to  be  non-linear,  users 
are  free  to  complete  the  program  on  their  on  schedule  and  in  the  order  they  desire.  Once  they 
complete  the  program,  they  are  taken  to  a  screen  congratulating  them  and  indicating  that  they 
have  completed  the  LSB  training  program  (Figure  A-24). 
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Figure  A-24.  Screenshot  of  “Certificate  of  Completion”  Page 
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